Non-invasive determination of disease states

ABSTRACT

An example method for estimating a disease state for a patient can include: capturing physiological data including images from the patient over time; processing the physiological data to detect a variability; comparing the variability to a baseline or a range; and estimating the disease state for the patient based upon the comparing of the variability to the baseline or the limit.

CROSS REFERENCE TO RELATED APPLICATIONS

The present application claims the benefit of U.S. ProvisionalApplication No. 62/395,427, filed Sep. 16, 2016 and claims the benefitof U.S. Provisional Application No. 62/440,967, filed Dec. 30, 2016,which is incorporated herein by reference.

BACKGROUND

Various procedures can be used to assess the health state of a patient.These procedures typically involve invasive tactics (e.g., gathering ofblood, biopsies, etc.) or, at a minimum, potentially unpleasantprocesses (e.g., assessment of cardiac performance over time using anECG machine with multiple leads connected to the patient) in order todetermine disease states. Patients sometimes avoid seeking medicaltreatment because of these unpleasant experiences.

SUMMARY

In one aspect, an example method for estimating a disease state for apatient can include: capturing physiological data including images fromthe patient over time; processing the physiological data to detect avariability; comparing the variability to a baseline or a range; andestimating the disease state for the patient based upon the comparing ofthe variability to the baseline or the limit.

DESCRIPTION OF THE FIGURES

FIG. 1 shows an example system for estimating a disease state of apatient using non-invasive procedures.

FIG. 2 shows an example medical device of the system of FIG. 1.

FIG. 3 shows another view of the medical device of FIG. 2.

FIG. 4 shows another example medical device of the system of FIG. 1.

FIG. 5 shows example logical components of the medical device of thesystem of FIG. 1.

FIG. 6 shows an example method for estimating a disease state of apatient using pulse rate variability.

FIG. 7 shows an example graph of pulse rate variability data capturedusing the method illustrated in FIG. 6.

FIG. 8 shows an example power/frequency plot of the pulse ratevariability data captured using the method illustrated in FIG. 6.

FIG. 9 shows an example spectrogram plot of the pulse rate variabilitydata captured using the method illustrated in FIG. 6.

FIG. 10 shows an example method for estimating a disease state of apatient using pupil diameter variability.

FIG. 11 shows an example method for estimating a disease state of apatient using physiological data captured over time.

FIG. 12 shows example components of a device of the system of FIG. 1.

FIG. 13 shows another example system for estimating vital signs fromfacial images.

FIG. 14 shows an example method for estimating pulse rate and/orrespiratory rate from video using the system of FIG. 13.

FIG. 15 shows additional example details on the application of anautoregressive model from the method of FIG. 14.

DETAILED DESCRIPTION

The present disclosure relates to medical devices that are used tocollect physiological data from patients. In the examples describedherein, the medical devices use non-invasive methods for assessingvarious disease states for a patient.

FIG. 1 is a block diagram illustrating an example system 100 forassessing the disease state of a patient 102 using non-invasiveprocedures. Disease includes any condition that causes pain,dysfunction, distress, social problems, or death to the afflicted. Thismay also include injuries, disabilities, disorders, syndromes,infections, isolated symptoms, deviant behaviors and atypical variationsof structure and function.

An example medical device 104 and/or mobile device 114 are used toassess the patient 102 using the non-invasive procedures. The results ofthe procedures can be analyzed by the medical device 104/mobile device114 and/or communicated through a network 108 to a server device 112 foranalysis and/or storage.

The analysis can result in the assessment of a disease state for thepatient 102, including a probability of having or contracting aparticular disease. Although only a single patient, medical/mobiledevice, and server device are shown, the number of each can be increasedor decreased as needed to scale the system 100. For example, multiplemedical or mobile devices can be used to process multiple patients, andthe data from the procedures can be analyzed by a server farm orcloud-based system to provide the disease state assessments.

The medical device 104 can be positioned in a facility, such as ahospital or clinic, at which the patient 102 is located. In anotherexample, the medical device 104 can be located at a facility that isremote geographically from the position of the patient 102 and/orcaregiver.

As noted, the medical device 104 communicates with the network 108. Inone example, the medical device 104 and the network 108 are part of aCONNEX™ system from Welch Allyn of Skaneateles Falls, N.Y., althoughother systems can be used. In such an example, the medical devicescommunicate through known protocols, such as the Welch AllynCommunications Protocol (WACP). WACP uses a taxonomy as a mechanism todefine information and messaging. Taxonomy can be defined asdescription, identification, and classification of a semantic model.Taxonomy as applied to a classification scheme may be extensible.Semantic class-based modeling utilizing taxonomy can minimize thecomplexity of data description management by limiting, categorizing, andlogically grouping information management and operational functions intofamilies that contain both static and dynamic elements.

The network 108 is an electronic communication network that facilitatescommunication between the medical device 104, the mobile device 114, andthe server device 112. An electronic communication network is a set ofcomputing devices and links between the computing devices. The computingdevices in the network use the links to enable communication among thecomputing devices in the network. The network 108 can include routers,switches, mobile access points, bridges, hubs, intrusion detectiondevices, storage devices, standalone server devices, blade serverdevices, sensors, desktop computers, firewall devices, laptop computers,handheld computers, mobile telephones, medical devices, and other typesof computing devices.

In various embodiments, the network 108 includes various types of links.For example, the network 108 can include wired and/or wireless links.Furthermore, in various embodiments, the network 108 is implemented atvarious scales. For example, the network 108 can be implemented as oneor more local area networks (LANs), metropolitan area networks, subnets,wide area networks (such as the Internet), or can be implemented atanother scale, such as a wide area network (WAN), body area network(BAN) and may include Internet of Things (IoT)/Internet of Healthcare(IoH) devices.

In this example, the medical device 104 and the network 108 are all partof the same network. In other words, the medical device 104 and thenetwork 108 communicate with one another over a LAN behind a firewallsafeguarding the devices from outside influences on the Internet, suchas a firewall. The mobile device 114 may or may not be configured tocommunicate within the network 108. Alternately, the medical device 104may communicate via the mobile device 114, for example using a WANconnection available on a cellular phone to reach the network 108. Insome instances, the medical device 104 may be instantiated as part ofthe mobile device 114.

As noted, the medical device 104 can provide various types offunctionality, including measuring or monitoring patient physiologicalparameters. The medical device 104 can include one or more physiologicalmonitor devices configured to measure and possibly displayrepresentations of one or more physiological parameters of a patient. Inaddition, the medical device 104 can include one or more desktop,laptop, or wall-mounted devices. In some embodiments, the medical device104 is configured to be used by a clinician to monitor the physiologicalparameters of multiple patients at one time. Such monitor devices aretypically not wall mounted.

In this example, the server device 112 is located “in the cloud.” Inother words, the server device 112 is located outside of the internalnetwork associated with the medical device 104, 104, 105. Typically, theserver device 112 does not communicate directly with the medical device104, but instead communicates with a central server located within thesame network as the medical device 104, such as the CONNEX™ system fromWelch Allyn of Skaneateles Falls, N.Y. Intermediary servers in theCONNEX™ system, in turn, communicate with the medical device 104. Otherconfigurations are possible.

The medical device 104 and the server device 112 are computing systems.As used herein, a computing system is a system of one or more computingdevices. A computing device is a physical, tangible device thatprocesses data. Example types of computing devices include personalcomputers, standalone server computers, blade server computers,mainframe computers, handheld computers, smart phones, special purposecomputing devices, and other types of devices that process data.

The mobile device 114 can be any computing device, such as a smartphone,smart watch, tablet, convertible, laptop, etc. that can be used tointeract with and capture data from the patient 102. In one example, themobile device 114 is a smartphone of the patient 102. Similar to thatnoted above, the mobile device 114 can communicate with the network 108to provide data captured from the patient 102 to the server device 112.

The mobile device 114 can be programmed to execute one or moreapplications, such as an application downloaded from the server device112 and installed on the mobile device 114. These one or moreapplications can be used to capture data from the patient 102, such aspulse rate, activity level, and/or images of the patient's skin orpupils. The applications can also be programmed to analyze that data,report the data to the server device 112, and/or alert the patient 102to various indicators, such as possible oncoming disease states. Otherconfigurations are possible. In addition to full-color images, imagesmay be monochromatic and associated with any wavelength, such as imagesderived from waves received in the range from 3-m to 300 nm. Images maycover any amount of the patient and surrounding area. Unobstructed viewis defined to be unobstructed at the wavelength of interest. Forexample, an instrument operation at 26 GHz would have an unobstructedview through bedding to image chest movement, even though at opticalfrequencies, the view is obstructed. View to inner ear, retina, etc.,may be achieved through optics familiar to those skilled in the art.

FIG. 2 illustrates one example of the medical device 104. The medicaldevice 104 is portable. The medical device 104 includes multiple healthcare equipment (HCE) modules. Each of the HCE modules is configured tomeasure one or more physiological parameters of a health-care recipient,also referred to herein as a patient. Other embodiments can include moreor fewer components than those shown in FIG. 2, or can include differentcomponents that accomplish the same or similar functions.

A temperature measurement module 212 is accessible from the front sideof the medical device 104. A SpO2 module 214 and a non-invasive bloodpressure (NIBP) module 216 are accessible from a left hand side of themedical device 104. An upper handle portion 220 enables the medicaldevice 104 to be carried by hand.

A front side of the medical device 104 includes a display screen 218 andan outer surface of the temperature measurement module 212. Thetemperature measurement module 212 is designed to measure the bodytemperature of a patient. As used in this document, a “module” is acombination of a physical module structure which typically resideswithin the medical device 104 and optional peripheral components (notshown) that typically attach to and reside outside of the medical device104.

The temperature measurement module 212 includes a front panel 212 a. Thefront panel 212 a has an outer surface that is accessible from the frontside of the medical device 104. The front panel 212 a provides access toa wall (not shown) storing a removable probe (not shown), also referredto as a temperature probe, that is attached to a probe handle 212 b. Theprobe and its attached probe handle 212 b are tethered to thetemperature measurement module 212 via an insulated conductor 212 c. Theprobe is designed to make physical contact with a patient in order tosense a body temperature of the patient.

A left hand side of the medical device 104 includes an outer surface ofthe SpO2 module 214 and an outer surface of the NIBP module 216. TheSpO2 module 214 is a HCE module designed to measure oxygen contentwithin the blood of a patient. The NIBP module 216 is a HCE moduledesigned to measure blood pressure of a patient.

As shown, the SpO2 module 214 includes a front panel 214 a. The frontpanel 214 a includes an outer surface that is accessible from the leftside of the medical device 104. The front panel 214 a includes aconnector 214 b that enables a connection between one or more peripheralSpO2 components (not shown) and a portion of the SpO2 module 214residing inside the medical device 104. The peripheral SpO2 componentsreside external to the medical device 104. The peripheral SpO2components are configured to interoperate with the SpO2 module 214 whenconnected to the SpO2 module 214 via the connector 214 b. In someembodiments, the peripheral SpO2 components include a clip that attachesto an appendage of a patient, such as a finger. The clip is designed todetect and measure a pulse and an oxygen content of blood flowing withinthe patient.

As shown, the NIBP module 216 includes a front panel 216 a having anouter surface that is accessible from the left side of the medicaldevice 104. The front panel 216 a includes a connector 216 b thatenables a connection between one or more peripheral NIBP components (notshown) and a portion of the NIBP module 216 residing inside the medicaldevice 104. The peripheral NIBP components reside external to themedical device 104. The peripheral NIBP components are configured tointeroperate with the NIBP module 216 when connected to the NIBP module216 via the connector 216 b. In some embodiments, the peripheral NIBPcomponents include an inflatable cuff that attaches to an appendage of apatient, such as an upper arm of the patient. The inflatable cuff isdesigned to measure the systolic and diastolic blood pressure of thepatient, the mean arterial pressure (MAP) of the patient, and the pulserate of blood flowing within the patient.

The medical device 104 is able to operate within one or more workflows(or profiles). A workflow is a series of one or more tasks that a userof the medical device 104 performs. When the medical device 104 operateswithin a workflow, the medical device 104 provides functionalitysuitable for assisting the user in performing the workflow. When themedical device 104 operates within different workflows, the medicaldevice 104 provides different functionality.

When the medical device 104 is manufactured, the medical device 104 isconfigured to be able to operate within one or more workflows. After themedical device 104 is manufactured, the medical device 104 can bereconfigured to operate within one or more additional workflows. In thisway, a user can adapt the medical device 104 for use in differentworkflows as needed.

In various embodiments, the medical device 104 operates within variousworkflows. For example, in some embodiments, the medical device 104 canoperate within a monitoring workflow or a non-monitoring workflow.Example types of non-monitoring workflows include, but are not limitedto, a spot check workflow, an office workflow, and a triage workflow. Anon-limiting example of a monitoring workflow is an intervals workflow.

FIG. 3 illustrates an example user interface displayed on the displayscreen 218 of FIG. 2. The medical device 104 outputs and displays userinterfaces discussed in this document on the display screen 218.

In some examples described herein, the physiological monitor device is aportable device. In other examples, the physiological monitor device isa non-portable device, such as a computing device like a workstation.Many configurations are possible.

The medical device 104 shown in FIGS. 2-3 is only one example of amedical device. All different types of medical devices used to collectpatient data can be used.

For example, another embodiment of the medical device 104 is shown inFIG. 4 on a mobile cart. In some examples, the medical device 104 can bea more compact device that includes a touch screen (e.g., 7 inches) andthe ability to execute multiple workflows.

The medical device 104 can be a portable device. In other examples, themedical device 104 can be a stationary device, such as computing deviceslike workstations. All different types of medical devices used tocollect patient data can be used. Many configurations are possible.

Referring now to FIG. 5, example logical components of the medicaldevice 104 are shown. The medical device 104 is programmed to include aparameter capture module 500, a parameter analysis module 502, and adisease state assessment module 504. The medical device 104 can beprogrammed to execute one or more of these modules 500, 502, 504 toperform a non-invasive assessment of a disease state of the patient 102,as described further herein. The parameter analysis module 502 and thedisease state assessment module 504 may be physically located on adifferent, external device, e.g., a cloud-based server. In anotherembodiment, some functions of the parameter analysis module 502 and thedisease state assessment module 504 may be present on the medical device104 and other functions on a remote device. For example, the modules onthe medical device 104 might be designed for low processing load, highsensitivity and low selectivity. When the output of these detects apossible disease state, the data are forwarded to the external devicefor additional analysis that is more selective.

In this example, the medical device 104 includes a parameter capturemodule 500. In further examples described below, the parameter capturemodule 500 is programmed to capture one or more parameters associatedwith the patient 102. In these examples, the parameter capture module500 captures these parameters in a non-invasive manner.

For example, the parameter capture module 500 can be programmed tocapture pulse rate and/or pupil dilation data over time using an imagingdevice. The first step in analysis is typically creation of a table orplot showing how the pulse rate or pupil diameter changes over time.Details on how this can be accomplished are provided below.

Once the one or more parameters are captured by the parameter capturemodule 500, the parameter analysis module 502 analyzes the captureddata. This analysis can take various forms. For example, the data can befiltered, extrapolated, and compared to baseline data to determinewhether changes have occurred.

Once the analysis of the captured data is complete, the disease stateassessment module 504 is programmed to estimate a disease state for thepatient 102. For example, the disease state assessment module 504 cananalyze the comparison of pulse rate variability data to a prior dataset and/or to typical data. Typical data may be focused based on patienthealth factors, and can include: patient's diagnosis, medical history,prescriptions, substance abuse, allergies, genetic data, fluids, andethnographic background such as age, gender, race and other data thatprovide a detailed, in-depth description of the patient's everyday lifeand practice. Typical data depends on the analysis type. For example,for the standard deviation of NN intervals (SDNN), the result is asingle number and comparison may be as simple as a Boolean expression.In contrast, a spectrogram is a picture where comparison requires imageanalysis. The comparison may be a scale where a number or range ofnumbers is compared to baseline values. For example, a decelerationcapacity of less than 2.5 milliseconds indicates a patient with a poorprognosis for cardiac recovery and a deceleration capacity greater than5 milliseconds indicates a patient with an excellent prognosis. Asliding scale might be used where 2.5-3.75 milliseconds indicates a fairprognosis and 3.75-5 milliseconds indicates a good prognosis. Medicalhistory and ethnography may be entered as parameters into the assessmentmodule 504 or retrieved from an external device, such as cloud serverdevice 112. Medical device 104 and Mobile Device 114 are anticipated tohave read/write access to the patient's medical record. Disease statemay be assessed in the cloud.

Through annotation of the data with patient outcomes such as morbidity,mortality, length of stay, transfers and other parameters that providemeasures of recovery, the data may be used to analyze algorithmperformance and provide a way to test new algorithms while minimizing aneed for clinical trials. Annotation may be achieved through review ofmedical records, clinician entry, EMR access or the like.

Referring now to FIG. 6, in one example embodiment, the medical device104 is configured to estimate a patient's pulse rate variability usingnon-invasive procedures. In this example, the medical device 104estimates the pulse rate variability using imaging and an estimated timeof the blood volume pulse.

Pulse rate is typically measured in beats per minute. The averageduration of time between heart beats is the inverse of the pulse rate.For example, the average duration of time between beats is one secondfor a pulse rate of sixty beats per minute (i.e., 60 seconds/60 beats).In reality, however, the duration of time between beats is not uniform.

Instead, pulse rate changes continuously in response to the body'sever-changing need for circulation. Pulse rate variability refers to thevariation in duration between beats of a person's heart. Within an agegroup, higher levels of pulse rate variability are typically associatedwith better health, perhaps indicating that the ability of the autonomicnervous system to respond to internal and external stimuli.

In this embodiment, the medical device 104 is configured (using forexample, a camera or other imaging device) to detect pulse ratevariability by imaging the skin of the patient 102. In this example, themedical device 104 images the skin at a high frame rate (e.g., 100-200fps) and uses the captured video/images to detect the patient's heartbeat and calculate pulse rate variability. Using a photodetector such asa photodiode illuminated by an LED, for example a 530 nm green LED,allows sampling of the green component of the skin color at sample ratesof at least 1 kHz using an SFH 7050 optical sensor from OSRAM OptoSemiconductors of Sunnyvale, Calif. and an AFE4044 integrated analogfront end from Texas Instruments of Dallas, Tex.

This information can then be presented to the caregiver in variousforms, such as trends, spectrograms, LaGrange plots, histograms, SDNN,root mean square of successive differences (RMSSD), low frequency/highfrequency (LF/HF) ratio, pulse rate variability triangular index,triangular interpolation of NN interval (TINN), differential index,logarithmic index, standard deviation of the averages of NN intervals(SDANN), SDNN index, standard deviation of successive differences(SDSD), number of pairs of successive NNs that differ by more than 50 ms(NN50) count, proportion of NN50 divided by total number of NNs (pNN50),total power, very low frequency (VLF power), LF power, HF power, LF/HFratio, α, etc. Analogous measures may be used for other measures of theparasympathetic nervous system, such as pupil diameter variability. Inaddition, the estimated pulse rate variability can be further informedusing other physiological data from the patient 102, such as oxygensaturation from an SPO2 sensor, from RADAR, from pressure sensors usingballistocardiography, from blood pressure measurements, video analysisand measurement of minute movements in the body due to pulsatile bloodflow, video analysis of skin color changes due to pulsatile blood flow,to or any other means by which heart beats or heart beat pulses aredetected.

Specifically, the imaging capability of the medical device 104 includesa camera that captures images that can be separated into the Red, Green,Blue (RGB) channels. By separating out the three color components, it ispossible to detect the pulse due to each heartbeat. Other color spacessuch as HSV may be used. The interval between pulses is measured andthen various algorithms are used to analyze the energy content andstatistical distribution of beat intervals. Pulse rate variabilityindices may be stored locally and/or transmitted to an electronic healthrecord. In either case, pulse rate indices may be used as an input to anearly warning score, as described further below.

Early warning scores are used to provide more timely assessments andpredictions to changes in patient acuity. SAPSII, APACHE, MEWS, PEWS,MEDS, REMS, ASSIST, and SCS are some of the many scoring systems thathave been adopted in emergency department (ED), medical/surgical,general care, and ICU environments.

The National Early Warning Score (NEWS) is based on a simple scoringsystem in which a score is allocated to physiological parameters thatare already recorded for patients in general care settings. Sixphysiological parameters form the basis of the scoring system:

-   -   Respiratory rate    -   Oxygen saturation    -   Temperature    -   Systolic blood pressure    -   Pulse rate    -   Level of consciousness (AVPU)

BTF is another Early Warning Score program focused on early recognitionof patient deterioration. This program builds on operational processesthat are in place and is not intended to take clinical assessment awayfrom the nurses. Nursing staff is expected to be able to have access toclinical data and to aggregate and analyze it to form a clinicalassessment. To help with this effort, the program established standardsoutlining what observations should be recorded and what thresholdsshould trigger a response. Instead of a numbering system, the protocolfocuses on human factors and the use of color.

These parameters form the basis of the scoring system:

-   -   Respiratory rate    -   Oxygen saturation    -   Temperature    -   Systolic blood pressure    -   Pulse rate

In example embodiments, the medical device 104, mobile device 114,and/or server device 112 are programmed to calculate early warningscores using one or more of the protocols described above. When doingso, the pulse rate variability information can be used to inform and/ormodify the early warning score. For example, if the pulse ratevariability information indicates a deterioration in the health of thepatient 102, the score can be modified as appropriate to indicate thisinformation. Alternately, the early warning score may be based purely onpulse rate variability or on pulse rate variability and a differentcombination of inputs than existing early warning scoring systems.

The devices 104, 114 also are programmed to communicate those scores,both visually to the caregiver, as well as possibly to a central server,such as server device 112 to be stored in an electronic medical records(EMR) system. Further, the devices 104, 114 are programmed to provideconfigurable alert messages based upon the calculated early warningscores. Medical device 104 and mobile device 114 are anticipated to haveread/write access to the patient's medical record.

Pulse rate variability indices may be used in conjunction withinformation other than early warning scores. For example, pre-termneonatal patients are susceptible to sepsis. Knowing this, the pulserate variability for the patient 102 is compared against a baseline fora healthy individual with similar demographics (e.g., neonatal) and, ifa significant variance is detected, a warning is provided to thecaregiver via the local display of the medical device 104, via theelectronic health record system, or a clinical alarm system.

An example method 600 for estimating the pulse rate variability of thepatient 102 is shown in FIG. 6. In this example, an image (e.g., videoimaging data) is captured of the skin of the patient 102 at operation602. Next, at operation 604, the imaging data is extracted into therespective RGB channels. Some devices, such as those with HDMI outputsand analog RGB outputs provide RGB as separate channels. In the formercase, software written to decode the communication protocol can separateout the various color streams. In the latter case, hardware such as theAD9983A from analog devices may be used to convert analog RGB intodigital streams. Color space convertors may be used to convert imageprocessing techniques from one color space to another, such as from as acolor space convertor may be used to convert between different colorspaces such as, YCbCr, HSV, CMYK, sRGB, Adobe RGB and the like. Thesystem might make a pulse rate analysis using multiple color spaces ormight determine which color space provide the optimum results andanalyze in that optimum color space.

At operation 606, interpolation is used on the extracted data todetermine the peaks of each pulse. The heart beat pulses are detected atoperation 608. This can be accomplished by multiple means, e.g., such asusing a matched filter that aligns with each pulse to determine a bestfit, familiar to those skilled in the art.

At operation 610, the data is smoothed to remove undesired aberrations,such as bad pulses caused by ventricular and non-physiologicalphenomenon. In one example, pulses have a variability greater than 10percent are excluded as being aberrant. Pulses determined aspre-ventricular contractions (PVCs) may be separated for a separateanalysis.

Next, at operation 612, the beat-to-beat intervals are measured, andthese intervals are analyzed at operation 614 using various techniqueslike SDNN (i.e., the standard deviation of NN interval, calculated for agiven period of time), spectrograms (as shown in FIG. 8 and FIG. 9, forexample), LaGrange plots, etc. At operation 616, the result is comparedto a baseline (either prior data from the patient 102 and/or data forsimilarly situated individuals). At operation 618, the information isused to calculate an early warning score. Finally, at operation 620, theresults are presented to the patient 102 and caregiver and/or saved inthe electronic health record.

In yet another example, other schemes are used to detect pulse rateother than visual procedures. For example, microwave and/orultra-wideband radio technologies can be used to detect pulse rate andthereupon variability. For example, a radar device can be incorporatedinto the medical device 104. The radar can be used to scan the chestproximal to the heart of the patient 102 and in-phase with the heartbeat to optimize detection of the atrial contraction and the ventricularcontraction. That is, the beam scans for and detects the location tomaximize the atrial response and it scans for and detects the locationto maximize the ventricular response.

Then, the system scans the radar beam toward the atria when an atrialcontraction is expected and follows it down to the ventricles when aventricular contraction is expected. The radar beam interacts with thepatient and some of the radiated electromagnetic signal isbackscattered. This backscattered signal is received by an antenna andthe sensor then demodulates data from the transmitted radar signal andtags the data so that it can be correctly associated with the patient102. This data can thereupon we used to calculate pulse rates andthereupon pulse rate variability. A radar may measure distance bysubtracting the pulse reception time from the pulse transmission time,e.g., distance_i=c/(Rx_time_i−Tx_time_i), where Rx_time_i is the timethe system received the i^(th) pulse, Tx_time_is the time the systemtransmitted the i^(th) pulse, and c is the speed of light. A radar maytransmit and receive a series of pulses and measure changes in distance,e.g. distance_change=distance_i+1−distance_i. A radar may also comparethe phase of the transmitted signal to the phase of the returned signal.

The phase difference, σ, between the transmitted and received signals isdirectly proportional to the distance, d, between the radar to thechest. σ=2*π*d/λ, where λ is the wavelength of the radar transmission.As the heart beats, the chest cavity presents a small deflection, on theorder of 0.1 mm, and this deflection can be detected as a change in thephase difference between the transmitted and received signal. Asexamples, continuous wave (CW) radar, pulsed radar, chirped radar,pulsed-CW, pulsed-chirped radar may all be used to detect the heartbeat.Detection of motion of the body, as a result of contraction of grossmuscles, may be done as a noise-detection-and-elimination step ofdetecting heart beats. For example, when a patient sits forward, thereis a large position change in the chest cavity compared to thesub-millimeter motion from a heartbeat. Detection of motion of the bodymay be used as a factor for estimating the likelihood that a patientwill contract pressure ulcers. Analysis of backscatter at opticalwavelengths may also be used to detect heart beat pulses, for example,by examination of video, one can detect minute head movements that aredue to pulsatile blood flow and also by examination of video, one candetect color changes in the skin due to pulsatile blood flow.

One may apply multiple measures of pulsatile inputs to improve thesystem sensitivity and specificity for correctly identifying when apulse occurs. Backscatter of pressure waves may be used to detectrespiration, heart-beat pulses, bladder state (full/empty) and the like.Reduced urine output may be the result of kidney failure, which in turnmay be due to sepsis, cholera, dehydration, etc. Pressure ulcers aremore likely for patients with soiled sheets; measures of urination anddefecation can be factors in the prediction of pressure ulcers. Theysystem may alert a clinician to detection of urination and/ordefecation.

Detection of body movement may also be used to detect a patient who mayattempt a bed-exit. For example, if the patient sits up and then movestoward the edge of the bed, the system may alert a clinician or modifythe bed settings.

Other optical methods may be used, such as a single or small number ofphoto detectors. The photodetector(s) may use back-scattered backgroundlight or light provided by a sensor. The light source may be broadband,e.g., white light, or it may be from a narrow-band optical source, suchas a 535 nm green light. The photodetector may be broadband, or filteredto preferentially detect light at the wavelength of a narrow-bandoptical source.

Referring now to FIGS. 7-9, the medical device 104 and/or the mobiledevice 114 are configured to illustrate aspects of the pulse ratevariability as measured per the method 600. The analysis methodsdescribed, including those for producing a spectrogram from pulse ratevariability, may be applied to other input sources, such as pupildiameter variability, respiration rate variability, temperaturevariability, blood pressure variability, and the like.

For pupil diameter variability, a graph similar to that shown in FIG. 7(for pulse rate variability) can be used, where the units are inmillimeters (as opposed to milliseconds for pulse rate variability). Forexample, the medical device 104 is configured to measure pupil size overtime and compare the size to a baseline, either of the patient 102and/or a similarly situated individual. This can be accomplished usingan imaging device (e.g., camera) of the medical device 104 and/or themobile device 114.

For example, a front facing camera of the mobile device 114 can beprogrammed to periodically capture images of the pupil(s) as the patient102 uses the mobile device 114 over time. A pupilometer may also beused. A pupilometer is a medical device that measures the diameter of apupil, typically in response to external stimulus, such as a lightturning on and off. Pupilometers such as the NeuroOptics NPi-200 measurethe response time to an externally forced light stimulus and measuressize, latency, constriction velocity and dilation velocity. There is nomeasure of how the pupil diameter varies in response to the autonomicbody regulation, such as respiration modulation; however, this could bedone as a source of data in this disclosure.

In some examples, the medical device 104 and/or mobile device 114provide a stimulus (e.g., various lighting configurations and/or videocontent) to impact the focus and dilation of the pupils. Once captured,the image data can be processed, and a spectrogram of the dilation overtime can be created. In the context of the mobile device 114, anapplication can be executed by the mobile device to periodicallystimulate, capture pupil images, and/or analyze/forward the images.

In some examples, a camera with a moderately high frame rate (≥16 fps)and algorithms to track and focus on the eye takes multiple images ofthe eye and measures the relative change in the pupil diameter overtime. The relative size of the pupil matters and other facialfeatures/landmarks such as the overall size of the eye can be used tore-scale if the distance to the camera changes. Images are taken, pupildiameters are measured, and a plot of pupil diameter versus time iscreated. From this plot, statistics such as mean and standard deviationcan be made. Groups of data, for example 128 points, can be transformedinto the frequency domain and then put together to create a spectrogram.

As with changes in beat-to-beat interval, the changes in pupil diameter,or pupil diameter variability, are reflective of the state of theautonomic nervous system. These changes can be due to varied factors inaddition to ambient light including: increased/decreased attention,surprise, seeing images, hearing sounds, etc. It is noted that theminute changes in pupil diameter are “rich” in healthy patients andrelatively plain in unhealthy patients. Respiration modulates the pupildiameter. For example, for a person breathing every 8 seconds, there isan 8-second periodicity in the pupil diameter.

In general terms, the process for calculating the pupil diametervariability involves multiple images of the eye, either using high-speedstill image capture or video imaging methods. Each image frame can beverified for quality (e.g., focus, lighting, ability to detect pupil &iris, etc.). The pupil diameter for each eye is measured for each frame.Heuristics are used to discard invalid data, e.g., pupil diameterchanges of over 50% from prior reading, pupil size>8 mm, etc. The pupilsize detector may use an infrared light and/or infrared detector as away to increase the contrast of the pupil compared to the iris.

A table of diameter versus sample size can be created. The tabular datamay be plotted on a graph similar to the graph 700, as shown in FIG. 7for pulse rate variability. Further calculations, such as statistics ofthe data (mean, standard deviation, histogram, etc.) are optionallycalculated. Frequency content of the tabular data is calculated, andfrequency content can be displayed graphically for a single group (partof all of the tabular data).

For example, FIG. 8 shows power being displayed both in color and in theamplitude on the y-axis of the plot 800. Multiple groups of data fromany physiological variability analysis can be displayed to show thetrend in frequency over time, as illustrated in the plot 900 of FIG. 9.Power can be compared to a previous baseline or to a general baselinefor patients with similar demographics as an indicator of health. Notein the plot 900 three general bands of power at about 0.03, 0.1, and 0.3Hz that are typical for pulse rate variability. Relative power indifferent bands can also be compared as indicators of some diseasestates. Different spectral bands and analysis methods may be used forpupil diameter variability and any other physiological inputs.

Alerts can be provided if the measured attribute (e.g., pulse ratevariability, pupil diameter variability, etc.) indicates highprobability of health issue, giving the caregiver an indicator thatperhaps a more detailed patient assessment is warranted, perhapsincluding an office visit.

Referring now to FIG. 10, an example method 1000 for measuring pupildiameter variability is shown. At operation 1002, imaging is initiated.In the context of a mobile device like a smartphone or tablet, optionaloperation 1004 can include displaying various images on the mobiledevice to orient and manipulate the pupil, such as video, video games,etc.

Next, at operation 1006, the pupils of the patient 102 are detected.This can be accomplished, for example, by using one or more automatedmethods for capturing the pupils of the patient 102. One such method foridentifying the pupil is described in U.S. Pat. No. 9,237,846 to Mowrey.

Next, at operation 1008, the one or both pupils are imaged at a givenrate, such as at 16 to 40 fps. For a human's typical maximum pupildiameter rate of change, sampling at 40 fps supports a sample forapproximately every 0.1 mm of diameter change. Higher frame rates of upto 600 fps would support measurement of diameter changes at a resolutionof 0.01 mm at the typical, fastest contraction rates of 6 mm/second.Each image can be qualified at operation 1010, such as for focus,pupil/iris detection, etc. At operation 1012, the relative pupildiameter is measured for each eye for each qualified frame.

Next, at operation 1014, the data is tabulated, such as in a table foreach eye. As needed, resampling can be done at operation 1016 dependingon the data captured. Next, at operation 1018, various statistics arecalculated using the data, and a frequency analysis can be performed atoperation 1020 analogous to methods for evaluating pulse ratevariability. As with pulse rate variability, the power and spectralbands can be analyzed at operation 1022, such as by using the plots 800and 900 shown in FIGS. 8 and 9. The results can be presented and/orstored locally or in the electronic health record at operation 1024.

For pulse rate, the sympathetic nervous system increases (accelerates)cardiac activity while the parasympathetic nervous system decreases(decelerates) cardiac activity. In comparison, dilation of the pupil iscontrolled by the sympathetic nervous system while contraction of thepupil is controlled by the parasympathetic nervous system. As such,analysis of the pupillary dilation/contraction provides a measure of theautonomic nervous system analogous to analysis of increases and decreasein pulse rate.

Referring now to FIG. 11, as noted, various physiological responses,such as pulse rate and/or pupil diameter variability, can be used overtime to predict disease states. Other physiological data, including butnot limited to, blood pressure variability, oxygen saturationvariability, blood glucose variability, temperature variability,respiration rate variability, respiration depth variability, tympanicmembrane motion variability, EEG variability, reflexive responsevariability, weight variability, urination/bladder-state, defecation, orany measure of variability in chemical content of the blood, urine,saliva or other bodily fluid, can also be used to estimate diseasestates.

For example, the medical device 104 and/or the mobile device 114 can beused to capture various physiological data of the patient 102 over time.Such data, like the variability of clinical parameters such as pulserate, respiration rate, SPO2 levels, temperature, pupil response and/orblood pressure can be used as early indicators of disease onset. Thisdata can be collected over time and analyzed to determine variabilitythat indicates a disease state.

As noted, pulse rate variability can be a strong predictor of disease.For example, research shows that pulse rate variability indicatorsprecede the onset of sepsis in adults by several (e.g., typically 1.5)days. A vital-sign variability algorithm can be implemented in a device(e.g., the medical device 104 and/or mobile device 114) that measures avital sign and provides an alert to the user or health-care worker bothwhen the variability decreases from baseline (indicating diseased state)and when the variability returns to the baseline. Variability may alsobe used as a health-check for home-health as being tired, stressed,over-exerted decreases the pulse rate variability baseline.

In FIG. 11, an example method 1100 shows how these variabilities can beused to estimate and/or predict disease states for the individual. Atoperation 1102, vital signs are captured using the medical device 104and/or the mobile device 114. For example, as noted, physiologicalinformation such as pulse rate variability (see FIGS. 6-9) and/or pupildiameter variability (see FIG. 10) can be captured using the camera ofthe medical device 104/mobile device 114.

Next, at operation 1104, baseline data is obtained. This data caninclude historic measurements for the patient 102. Or, the baseline datacan be obtained from a central repository (e.g., on the server device112) for an individual who is similarly situated (e.g., same gender,age, race, nationality, geographic origin, diagnosis, etc.).

At operation 1106, a spectrogram and other variability measures areproduced based upon the data, such as that shown in FIG. 9. Otherstatistics can also be used. At operation 1108, the results are comparedto the baseline data (either specific to the patient 102 and/ordemographic).

Next, at operation 1110, the physiological data is captured over time,and the new data is again analyzed at operation 1112. At operation 1114,significant changes are noted. If no significant changes are noted,control is passed back to operation 1110 for further capture of data.Trends in the changes may be noted and considered as part of theanalysis.

If, however, a significant change to the variability is noted, controlis instead passed to operation 1116, where a determination is made as towhether the change is positive or negative. If the change is positive,then control is passed to operation 1118 for an optional indication thatthe disease state is improving, and then control is passed to operation1110 for further capture of data. If, instead, the change is negative,control is passed to operation 1120, and some type of alert can begenerated for the patient 102 and/or the caregiver. Various methods ofindicating improvement or decline may be implemented, such as a Likertscale, color coding, audible alerts, and the like.

FIG. 12 illustrates example physical components of a computing device,such as the medical device 104, server device 112, and/or mobile device114. As illustrated, the device includes at least one processor orcentral processing unit (“CPU”) 1208, a system memory 1212, and a systembus 1210 that couples the system memory 1212 to the CPU 1208. The systemmemory 1212 includes a random access memory (“RAM”) 1218 and a read-onlymemory (“ROM”) 1220. A basic input/output system containing the basicroutines that help to transfer information between elements within thedevice, such as during startup, is stored in the ROM 1220. The devicefurther includes a mass storage device 1214. The mass storage device1214 is able to store software instructions and data. The centralprocessing unit 1208 is an example of a processing device.

The mass storage device 1214 is connected to the CPU 1208 through a massstorage controller (not shown) connected to the bus 1210. The massstorage device 1214 and its associated computer-readable data storagemedia provide non-volatile, non-transitory storage for the device.Although the description of computer-readable data storage mediacontained herein refers to a mass storage device, such as a hard disk orCD-ROM drive, it should be appreciated by those skilled in the art thatcomputer-readable data storage media can be any availablenon-transitory, physical device or article of manufacture from which thedevice can read data and/or instructions. The mass storage device 1214is an example of a computer-readable storage device.

Computer-readable data storage media include volatile and non-volatile,removable and non-removable media implemented in any method ortechnology for storage of information such as computer-readable softwareinstructions, data structures, program modules or other data. Exampletypes of computer-readable data storage media include, but are notlimited to, RAM, ROM, EPROM, EEPROM, flash memory or other solid-statememory technology, CD-ROMs, digital versatile discs (“DVDs”), otheroptical storage media, magnetic cassettes, magnetic tape, magnetic diskstorage or other magnetic storage devices, or any other medium which canbe used to store the desired information and which can be accessed bythe device.

According to various embodiments of the invention, the device mayoperate in a networked environment using logical connections to remotenetwork devices through the network 108, such as a local network, theInternet, or another type of network. The device connects to the network108 through a network interface unit 1216 connected to the bus 1210. Thenetwork interface unit 1216 may also be utilized to connect to othertypes of networks and remote computing systems. The device also includesan input/output controller 1222 for receiving and processing input froma number of other devices, including a camera, a keyboard, a mouse, atouch user interface display screen, or another type of input device.Similarly, the input/output controller 1222 may provide output to atouch user interface display screen, a printer, or other type of outputdevice.

The device also includes an imaging device 1230, such as a camera thatis configured to capture still or moving images (i.e., video). Thecamera can be configured to capture high resolution images or video(e.g., 100-200+ fps) that can be used to conduct the analyses describedherein.

As mentioned above, the mass storage device 1214 and the RAM 1218 of thedevice can store software instructions and data. The softwareinstructions include an operating system 1232 suitable for controllingthe operation of the device. The mass storage device 1214 and/or the RAM1218 also store software instructions, that when executed by the CPU1208, cause the device to provide the functionality of the devicediscussed in this document. For example, the mass storage device 1214and/or the RAM 1218 can store software instructions that, when executedby the CPU 1208, cause the medical and/or mobile device to captureimages and determine variability of one or more physiologicalmeasurements.

Referring now to FIGS. 13-15, another embodiment of a system isillustrated for estimating vital signs, like blood volume changesassociated with the cardiac cycle, from facial images of human subjects.Patients' vital signs, like pulse rate and respiratory rate, can bemonitored continuously by caregivers, which can be useful for patientssuffering respiratory disorder diseases and heart problems. This exampleprovides a low-cost and non-contact solution for patients who have adifficulty using contact and wearable devices.

In this example shown in FIG. 13, a system 1300 includes a camera 1310that is placed to capture a sequence of images of the patient 102. Theimages are segmented into various regions (e.g., face, upper body andbackground). The camera 1310 may be a still camera that acquires imagesat suitable frame rate (e.g., 10-16 images per second) or a videocamera.

Subtle changes in pixel values of the facial region (e.g., forehead andcheek) are used to estimate pulse rate 1325, as depicted in the waveform1320. In addition, low frequency amplitude variation of the camerareflectance signal can be used for estimating respiratory rate. Thepulse and/or respiratory rates can be estimated and recordedcontinuously. If abnormal rates are detected, an alert 1330 is triggeredand sent to a caregiver 1340 in order to take proper action, if needed.The trends of pulse and respiratory rates will also be available for thecaregiver 1340 to analyze. The system may use combine multiple analysismethods, such as the changes in pixel values and the minute motions ofthe head occur with each pulse to achieve a higher fidelity system thanone that uses just one of the methods.

Referring now to FIG. 14, an example method 1400 for estimating pulserate and/or respiratory rate from video is shown. In this example, thevideo is captured at operation 1410 as a 12 Hz RBG video feed, althoughother configurations can be used.

Next at operation 1420, the captured video feed is broken into red,green, and blue components and an average region of interest (ROI) iscalculated. At operation 1430, a sliding window of a defined size (e.g.,a 30 second window in this example) is used to analyze the data.

At operation 1440, the data is filtered (e.g., detrended and filtered).The filtering can be altered depending on the desired data, such asdifferent filtering for pulse rate and respiratory rate.

Next, at operation 1450, the data can be downsampled. At operation 1460,an autoregressive (AR) model is applied, as described further below.Finally, at operation 1470, an estimate of pulse rate and/or respiratoryrate are provided. This estimate can be captured over time to providetrending for the caregiver. In addition, acute alerting can be providedif significant changes are captured.

Referring now to FIG. 15, additional details on the application of theAR model in operation 1460 of the method 1400 for spectrum estimation ofpulse rate and/or respiratory rate are provided.

Generally, in this example, the time series of data collected by thecamera is digitized at operation 1510 to form a photoplethysmogram (PPG)waveform. Next, the data is modeled to form the AR model at operation1520.

At operation 1530, the AR coefficients are calculated using an equationsuch as the Yule-Walker equations. Next, at operation 1540, frequencyscanning is used to obtain the discrete AR spectrum. Finally, atoperation 1550, the peak frequency for pulse rate or respiratory rate isestimated.

More specifically, the AR model of the digitized time series data (PPGwaveform) can be expressed as follows.

${{x(n)} + {\sum\limits_{i = 1}^{m}\;{a_{i}{x\left( {n - i} \right)}}}} = {e(n)}$where m is the order of the AR process, {a_(i)}_(i=1) ^(m) representsthe AR coefficients, e(n) is assumed to be the white noise with zeromean and variance σ². In the domain of the z-transform, the transferfunction H(z) relating the output to the input can be written asfollows.

${H(z)} = \frac{1}{A(z)}$where

${A(z)} = {\sum\limits_{k = 0}^{m}\;{a_{k}z^{- k}}}$and z⁻¹ is the unit delay operator, that is, z^(−k)x(n)=x(n−k). Then,the power spectral density (PSD) of the AR model can be expressed asfollows.

${P(\omega)} = {\sigma^{2}{\left( \frac{1}{A(\omega)} \right)}^{2}}$${{{where}\mspace{14mu}{A(\omega)}} = {\sum\limits_{k = 0}^{m}{a_{k}e^{{- j}\; k\;\omega}}}},$and A(ω) is obtained by substituting z=e^(jω) in A(z). As noted, the ARcoefficients can be obtained by the Yule-Walker equations.

After obtaining the AR coefficients, the frequency scanning process canbe performed to find the peak frequency locations for a range offrequencies of interest. As such, the AR frequency spectrum is obtainedin order to estimate the frequency components of the given digitizedtime series data. The frequency ƒ of each peak can be obtained from theangular frequency ω=2πƒ. Measurements over time may be presented to theclinician through various means including display of current data and/ortrends on the medical device 104, mobile device 114, or via serverdevice 112, which allows clinicians to view medical data via an EHRinterface.

Although various embodiments are described herein, those of ordinaryskill in the art will understand that many modifications may be madethereto within the scope of the present disclosure. Accordingly, it isnot intended that the scope of the disclosure in any way be limited bythe examples provided.

What is claimed is:
 1. A method for estimating a disease state for apatient, the method comprising: capturing images of an unobstructedportion of the patient; processing the images to detect a pulse rate;determining a pulse rate variability based upon the pulse rate; andcalculating an early warning score using the pulse rate variability,including: comparing the pulse rate variability to a baseline havingsimilar demographics; and increasing or decreasing the early warningscore based upon a variance from the baseline.
 2. The method of claim 1,further comprising transmitting pulse rate variability to an electronicmedical record.
 3. The method of claim 2, further comprising annotatingdata with at least one patient outcome metric.
 4. The method of claim 3,further comprising using one or more annotations to measure algorithmperformance.
 5. The method of claim 1, wherein estimating the diseasestate further comprises analyzing patient health factors.
 6. The methodof claim 1, further comprising processing the images to detect arespiratory rate for the patient.
 7. The method of claim 1, furthercomprising forming an autoregressive model using data from the images toestimate the pulse rate of the patient.
 8. The method of claim 7,further comprising combining the autoregressive model estimate of thepulse rate and pulse-rate estimates derived from analysis of motiondata.
 9. The method of claim 1, further comprising providing a trend ofthe pulse rate variability for the patient over time.